rewrite nixpkgs's create-amis script using Terraform #254
Conversation
AmineChikhaoui
force-pushed
the
terraform-create-amis
branch
from
e75d7a5
to
5c3859c
Compare
AmineChikhaoui
force-pushed
the
terraform-create-amis
branch
from
f0172c1
to
1a20608
Compare
|
@zimbatm @grahamc if you can do a quick review or if there are any suggestions. AWS OIDC setup is pointing to my fork/branch but can switch it before merging to nixos/nixos-org-configurations. I can also terraform the IAM role config for visibility. Here is an example of the workflow runs: https://github.com/AmineChikhaoui/nixos-org-configurations/actions/runs/5172652543 |
|
Something that I just discovered is the Though I don't know if the cross-region copying means you'd need a provider for each region (as is currently necessary). I also don't know if the fact that it's for managing a lifecycle makes it unsuitable for this, though (i.e. if it requires you to specify something that would "clean up" images / etc and not just use it for cross-region copying). EDIT: But the fact the documentation says "for the copied snapshot", "cross-Region snapshot copies", etc makes me think that this might not work exactly as I originally thought 🥲 EDIT2: There is also this |
That's the one I'm using in regions.jq right ? |
|
......yes. 🤦 If only there was some easier way to do this :') |
|
@cole-h probably CDK but not sure if it's worth the effort compared to a few lines of I'll look into DLM, I was planning to explore it for managing the deprecation and removal of AMIs. Didn't see copy AMI handling specifically from a quick scroll over docs but can look a bit more. |
The release management process involves uploading functional AMIs at each release for NixOS. As this script mentions, the 'entire thing' should be replaced with a new setup involving Terraform. Such a new setup can be found in NixOS/infra#254. In the meantime, this script should not be used anymore until a better solution for AMI uploading is found. This means effectively that AMI uploading is not part of the release management process for further releases. Feel free to push forward the PR mentioned to get back AMI upload.
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: |
|
Trying to understand the status of this PR - what is the relation between this work and the (currently operational, I think) https://github.com/NixOS/amis maintained by @arianvp? Are these complementary / parallel implementations / does one deprecate the other / ...? (Bear with me, I have very little context, but I'd like to clean up this repo and make sure we have a clear set of pending items to look at :p) |
|
@delroth I didn't read through https://github.com/NixOS/amis yet but my understanding it will deprecate this process and this PR won't be needed anymore in that case. |
This rewrites create-amis.sh using terraform.
The create-amis.sh was pretty slow and switching to Terraform makes things more declarative and easier to read.
There are few things I don't like in the Terraform code, since we're copying the AMI to a set of regions (which require a different aws provider per region), the module system of terraform doesn't really work for this. The easy solution was to just use
jqandmaketo generate the copy ami resources in JSON spec instead of HCL.Even using Terraform, this still takes 15minutes which isn't that fast either but at least ami copying is done in parallel.
A note about usage of Terraform workspaces, since we don't want to trigger terraform replacements i.e delete and recreate of resources as we want to keep AMIs being used (and perhaps use a lifecycle to purge them periodically), I went with a workspace (statefile) per
{release}.{arch}.{buildId}.Things I'm planning to explore now that the building blocks are here: